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Abstract 

Objectives The population of randomized experiments in policing is used to examine 
co-author and mentoring relations in the professional network of scholars and assess 
if experimental criminology is on the path to creating the necessary social capital to 
promote the use of randomized controlled trials in criminology and criminal justice 
research. 

Methods We use systematic review methods to identify the population of policing 
experiments. Narrative review and descriptive statistics are used to examine the 
growth of policing experiments over time. Social network analysis techniques are 
used to analyze and describe the co-authoring and mentoring connections of the 
scholars responsible for completing policing experiments. 

Results We find that the number of policing experiments increased substantially 
between 1970 and 2011. The growth in policing randomized experiments has been 
largely generated by a very small number of scholars who account for the bulk of 
policing experiments and have been very active in mentoring the next generation of 
experimentalists. Another important factor associated with the rise in policing exper- 
iment is the availability of federal funding. 
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Conclusions Our analysis of policing experiments suggests that the experimental 
criminology movement is developing the necessary human and social capital to 
advance the discipline of criminology. However, it is a very small network that could 
benefit from the addition of new members and increased training and mentoring of 
graduate students. 


Keywords Policing - Randomized experiments - Mentoring - Social network 


Introduction 


The randomized experiment is generally considered the strongest research design 
available to evaluate programs and test theories due to its strong internal validity 
(Shadish et al. 2002; Berk 2005). When implemented properly, randomized experi- 
ments provide the clearest assessment of causation. Experimental criminology is a 
part of a larger and increasingly expanding evidence-based movement in social 
policy. In general terms, this movement is dedicated to the improvement of society 
through the utilization of the highest-quality scientific evidence on what works best 
(see, e.g., Sherman et al. 1997). The evidence-based movement first began in 
medicine and has, more recently, been embraced by the social sciences. Leading 
experimental criminologists and organizations, such as the Academy of Experimental 
Criminology and the Campbell Collaboration’s Crime and Justice Group, have been 
strong advocates for the advancement of evidence-based crime policy and the use of 
randomized experiments in criminology. 

The number of randomized experiments in criminology has grown considerably 
over the last three decades (Farrington and Welsh 2005; Welsh et al. 2013). Unfor- 
tunately, relative to quasi-experiments and observational studies, the overall number 
of randomized experiments in criminology remains small. Randomized experiments 
can be labor intensive and require scarce resources such as funding, organizations 
willing to modify their operations to test programs, and the availability of skilled 
experimental criminologists. While other elements are clearly important, we believe 
that social capital, in the form of skilled experimental criminologists and a robust 
professional network, is a central consideration to the continued growth of random- 
ized experiments in the field. Ensuring the integrity of the execution of randomized 
experiments under field conditions can be very difficult and implementation problems 
represent the biggest threat to the internal validity of the design (Sherman 2010; 
Weisburd and Hinkle 2012). Well-trained and experienced experimental criminolo- 
gists can ensure design integrity by anticipating and managing implementation 
problems. 

In this paper, we examine the professional network of scholars who have com- 
pleted randomized controlled trials in policing. We use systematic review methods to 
identify the population of randomized experiments in policing and social network 
analysis techniques to describe the relationships among co-authors of existing polic- 
ing experiments. Mentoring connections among policing experimentalists are also 
examined to assess the salience of graduate student training in the continued growth 
of policing experiments. We find that a very small number of tightly networked 
experimental criminologists are responsible for a large proportion of the growth in 
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policing randomized experiments between the 1970s and the 2000s. Further, we find 
that a “vital few” experimental criminologists are also responsible for mentoring the 
next generation of experimentalists, who have also made substantive contributions to 
the growth of randomized controlled trials in policing. Implications for the continued 
expansion of experimental criminology are discussed in the concluding section. 


Randomized experiments in criminology 


There are strong theoretical and practical advantages for using randomized experi- 
ments to evaluate crime prevention and criminal justice programs (Farrington 1983; 
Weisburd and Hinkle 2012). The key feature of randomized experiments is that the 
random assignment equates the experimental and control groups before the experi- 
mental intervention on all possible extraneous variables that might influence the 
outcome (e.g., crime). Hence, any subsequent differences between the groups must 
be attributable to the intervention. Randomization is the only method of assignment 
that controls for unknown and unmeasured confounders as well as those that are 
known and measured (Weisburd et al. 2001). However, the randomized experiment is 
only the most convincing method of evaluation if it is implemented with full integrity. 
To the extent that there are implementation problems (e.g., problems of maintaining 
random assignment, differential attrition, crossover between control and experimental 
conditions), internal validity could be reduced. 

Another important feature of the randomized experiment is that a sufficiently large 
number of units (e.g., people, areas) need to be randomly assigned to ensure that the 
treatment group is equivalent to the control group on all extraneous variables (within 
the limits of statistical fluctuation). In their review of randomized experiments on 
crime and justice, Farrington and Welsh (2005, 2006) argued for a minimum sample 
size of 100 units; that is, at least 50 units had to be initially assigned to the 
experimental and control conditions. However, this is not a hard-and-fast rule, and 
some research suggests that a smaller n may not compromise the equivalence of the 
conditions on extraneous variables (Gill and Weisburd 2013). 

Other things being equal, an intervention study in which the experimental and 
control units are matched or statistically equated (e.g., using a prediction score) prior 
to intervention—what is called a nonrandomized experiment—has lower internal 
validity than a randomized experiment. An intervention study with no control group 
has even less internal validity since it fails to address many threats to internal validity, 
such as history, maturation, regression to the mean, and testing or instrumentation 
effects (Cook and Campbell 1979). 


Growth of randomized experiments 


The use of randomized experiments in criminology is characterized by an upward 
trend over the last six decades. In Farrington’s (1983) seminal review on the subject, 
he identified 37 trials published in English between 1957 and 1981. An additional 85 
randomized experiments were published between 1982 and 2004 (Farrington and 
Welsh 2006). Using a broader set of criteria for inclusion of studies (e.g., no 
minimum sample size, unpublished reports), Petrosino et al. (2003) identified 267 
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randomized experiments in criminology carried out between 1945 and 1993. The 
annual average number of experiments increased sharply from 1.8 in 1961-65 to 9.4 
in 1971-75 and 1986-90 to 11.6 in 1991-93. By many accounts, there has been a 
continued increase in the number of randomized experiments in criminology over the 
last several years (Welsh et al. 2013; Welsh and Farrington 2012). 

A number of methodological advances have contributed to this growth. Between 
the two periods of 1957-81 and 1982-2004, there was a marked increase in the 
number of large-scale, multisite replication experiments as well as an increased 
number of experiments (especially on prevention) with long-term follow-ups. The 
most noteworthy methodological advance, however, was the increased use of places 
as the unit of assignment (Farrington and Welsh 2006). This was especially the case 
in the area of policing, and helped to establish what is now referred to as place-based 
randomized experiments (Sherman and Weisburd 1995; see also Braga and Weisburd 
2010; Weisburd 2005). 

Despite this state of affairs, randomized experiments continue to be the exception 
rather than the rule in evaluating crime prevention and criminal justice programs. One 
measure of their relative use comes by way of Weisburd et al.’s (2001) analysis of the 
Maryland report on the effectiveness of criminological interventions (Sherman et al. 
1997). Of the 308 studies with offending outcomes that Weisburd et al. (2001) 
analyzed, only 46 (14.9 %) were classified as randomized experiments. Another 
indicator of the small fraction of criminological evaluations that employ the exper- 
imental method is evident among systematic reviews published by the Campbell 
Collaboration. For example, only eight out of 154 (5.2 %) evaluations of drug courts 
(Mitchell et al. 2012) and four out of 32 (12.5 %) evaluations of correctional boot 
camps (Wilson et al. 2008) used randomized experiments. There are exceptions. In 
their updated systematic review of hot spots policing, Braga et al. (2012) found that 
ten out of 19 evaluations (52.6 %) used randomized experiments. 


Key challenges 


Part of the critique of randomized experiments—why some view them as the 
“bronze” standard or worse rather than the “gold” standard of evaluation design 
(Berk 2005; Sampson 2010)—has to do with the messy nature of conducting them in 
real-life or field settings. As noted above, randomized experiments are not immune to 
myriad implementation problems. Take the problem of differential attrition, for 
example. Researchers are acutely aware, if not obsessive, about the need to avoid 
the loss of subjects as an evaluation progresses from baseline measurement to 
intervention to post-intervention follow-ups. This is compounded in the case of 
randomized experiments because of the added concern with differential attrition, 
whereby, there is a differential loss of units (e.g., people) from experimental com- 
pared to control conditions. Significant differential attrition presents a serious threat 
to the integrity of the experiment because the benefits of randomization are 
compromised. Randomizing within matched pairs is one way to avoid the problem 
of differential attrition. 

Many other challenges confront the conduct of randomized experiments, and these 
may go some way toward explaining why randomized experiments remain relatively 
underused in criminology. Some of these challenges include: availability of funding; 
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willingness of organizations to participate and, importantly, for program administrators 
to relinquish control of the assignment procedures to research staff; legal and ethical 
considerations; and even the availability of skilled researchers to carry them out. 

It is the latter issue that is the starting point for this paper, and one that we view to 
be especially important for the continued advancement of the experimental method 
and its contribution to more rational and effective crime policy. For sure, it makes no 
difference how many skilled (and interested) researchers there are in conducting 
randomized experiments unless many of the other challenges can be overcome. The 
good news is that six decades of randomized experiments in criminology show that 
these challenges can be overcome. 


The availability of skilled experimental criminologists 


The advantages of experimental methods in testing theories of crime causation and 
prevention help to explain why the number of randomized controlled trials has grown 
considerably over the last three decades (Farrington 1983; Farrington and Welsh 
2006). However, these advantages depend on the capability of experimenters to 
ensure that the necessary elements of an unbiased comparison are achieved. Sherman 
(2010) observed that many randomized experiments in criminology suffer from flaws 
that could have been avoided with better planning by the experimenters. He further 
States: 


The lack of such planning, in turn, may be due to the scant attention paid to 
field experiments in research methods’ texts and courses. Even skilled, senior 
researchers can make basic mistakes when conducting field experiments, since 
experiments require a very different set of skills and methods than the “normal 
science” of observational criminology. As in any complex work, the value of 
10,000 hours of practice can make an enormous difference in its success... 
(Sherman 2010: 399). 


Experimenters need to be very knowledgeable in the necessary steps and preferred 
decisions required to plan, conduct, complete, analyze, report, and synthesize high- 
quality randomized controlled trials (Sherman 2010). 

There are, unfortunately, only a small number of experimentalists in the broader 
field of criminology. Similar to the skewed distributions of income, land ownership, 
and criminal behavior (commonly known as the Pareto Principle or the “80-20 rule”; 
see Juran 1951; Sherman et al. 1989a, b), there are a “vital few” criminologists who 
are responsible for the promotion and use of randomized experiments as an important 
methodology in crime and justice studies. At present, slightly more than 7 % (n=191) 
of the American Society of Criminology’s (ASC) estimated 2,900 members (Miller 
and Brunson 2011) are also members of its Division of Experimental Criminology 
(DEC).' The DEC membership ranges from graduate students who are learning 
experimental methods to interested scholars with modest field experience in exper- 
imentation to seasoned experimentalists. One important avenue to increase the 
number of randomized experiments in criminology and criminal justice is to increase 


' http://gemini.gmu.edu/cebep/dec.html 
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the number of criminologists who are trained in experimental methods and have 
experience in implementing randomized experiments in field settings. 

The mentoring of graduate students by experienced experimenters is a critical 
mechanism to increase the number of skilled experimental criminologists available to 
conduct randomized experiments. Mentors and mentoring can have many meanings 
and connotations depending on different fields of study and among various authors 
writing on the subject (e.g., see Brustman 1991). Here, we define a mentor as “a 
person who leads, guides, and advises someone more junior in experience toward 
career accomplishments” and mentoring as “the process by which the protégé is 
guided, taught and influenced” (Anderson and Ramey 1990: 183-184). Mentors 
develop protégés by teaching graduate students experimental methods in the class- 
room and involving them in the design and execution of field experiments. 

There is some prima facie evidence that mentorship matters in generating random- 
ized experimental research. In an empirical analysis of experimenters in the Maryland 
report, Lum and Yang (2005) found that two-thirds of the researchers who had 
completed randomized controlled trials had students and colleagues they worked 
with go on to conduct other randomized experiments. By comparison, less than a 
third of the researchers who had not conducted randomized experiments had students 
and colleagues go on to conduct randomized experiments on their own. 

The advancement of experimental criminology is dependent on developing and 
expanding a steady supply of human capital. Thomas Kuhn (1962) noted that the 
potential for shifting paradigms in science rested, in part, on the ability of a specific 
scientific sub-community to gather human resources to act as a vanguard for a new way 
of thinking or doing science. Revolutionary ideas and new theoretical perspectives 
cannot move forward if converts are not drawn to the cause. Graduate students can be 
trained to be the “worker bees” in such revolutionary movements (Ballard et al. 2007: 
286). Merton (1968) suggests that successful scientists pass along their unique ways of 
doing science, their knowledge, experience, and expertise to future generations. Sum- 
marizing the perspectives of Kuhn and Merton, Ballard et al. (2007) observe that 
protégés maintain the heritage of methods, systems of thought, and processes of success 
they were taught during their graduate training. A robust professional network of 
scholars in a specific sub-community, such as experimental criminology, should be 
characterized by mentor-protégé connections between generations of scholars. In addi- 
tion to human capital, experimentalists can leverage social capital—the investment they 
make into their social networks and relationships, and how their placement within key 
positions in social networks can be used to advance experimental criminology. 


Policing experiments as a unique example 


The increase in the absolute number of randomized experiments in criminology over 
time has been especially pronounced in policing. Between the two periods of 1957— 
81 and 1982-2004, policing experiments increased from 4 to 12 (Farrington and 
Welsh 2006). It bears repeating that the latter period represents an undercount of the 
number of policing experiments, because not all place-based randomized experiments 
were included. More recent years suggest that this growth in policing experiments has 
continued. For instance, the Evidence-Based Policing Matrix recently identified 29 
policing experiments (Lum et al. 2011b). 
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This growth is one reason for our focus on analyzing the professional network of 
experimental criminologists in the area of policing. Another reason is that police officers 
follow orders regarding how and where to exercise their discretion; the high degree of 
hierarchical control in police agencies makes it easier to implement randomized exper- 
iments (Weisburd and Hinkle 2012). Yet another reason is what appears to be a unique 
openness and willingness on the part of police leaders across the country—in large and 
medium-sized cities—to engage the experimental method. Indeed, police leaders have 
been at the forefront in embracing the need for more experimentation in criminal justice 
and promoting greater academic-practitioner relationships (Braga and Hinkle 2010). 
More generally, Petersilia (2008) suggests that policymakers and practitioners today are 
often willing to support randomized experiments and are more likely to be influenced by 
experimental findings than in the past. 


Systematic review method and results 


Our examination of the growth of randomized experiments in policing used the system- 
atic review protocols and conventions of the Campbell Collaboration (Farrington and 
Petrosino 2001). Systematic reviews use rigorous methods for locating, appraising, and 
synthesizing evidence from prior evaluation studies, and they are reported with the same 
level of detail that characterizes high quality reports of original research. In this study, 
we used the systematic search methodology to identify the population of randomized 
policing experiments. 


Criteria for inclusion and exclusion of studies 


Eligible studies had to use a randomized controlled trial research design (Cook and 
Campbell 1979; Campbell and Stanley 1966). All identified studies were closely 
screened to ensure that the evaluation, as initially designed and implemented, used a 
randomized controlled design. Some well-known evaluations described in the polic- 
ing literature as experiments were excluded because units of analysis were not 
randomly allocated to treatment and control conditions. For instance, the landmark 
Kansas City Preventive Patrol Experiment was excluded from this review because the 
15 study beats were statistically matched and allocated based on optimal patrol route 
considerations rather than randomly (Kelling et al. 1974). Conversely, we did include 
evaluations that were designed and executed as randomized controlled trials but 
suffered from modest implementation problems that threatened the integrity of the 
initial design. For example, the Minneapolis Domestic Violence Experiment was 
designed to randomly assign arrest, separation, and some form of advice which could 
include mediation at the officer's discretion to misdemeanor domestic assault inci- 
dents (Sherman and Berk 1984). Some participating officers failed to follow fully the 
randomization plan and supplemental analyses were necessary to correct for possible 
biases introduced by violations of the random assignment. 

Eligible studies also had to have treatments that involved police departments in a 
central programmatic role. It was permissible for treatments to involve additional 
partners. For instance, restorative justice evaluations that reported police involvement 
in the face-to-face meetings among crime victims, their offenders, and their respective 
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families and friends were considered (e.g., Strang et al. 1999). Restorative justice 
evaluations of court-diversion programs that did not involve the police in the face-to- 
face meetings were not considered. Finally, to be eligible for this review, treatments had 
to involve police programs implemented in field settings. Evaluations conducted in 
laboratory settings, such as a clinical test of less-than-lethal weapons on human stress 
response (e.g., Dawes et al. 2009), were excluded. 


Search strategies for identification of studies 


Several strategies were used to perform an exhaustive search for evaluations meeting 
the eligibility criteria. First, a keyword search” was performed on 17 online abstract 
databases.* Second, the bibliographies of past narrative and empirical reviews of 
literature that examined evaluations of police programs (Lum et al. 201 1a; Sherman 
2002; Skogan and Frydl 2004; Weisburd and Eck 2004) and, more generally, the use 
of experiments in crime and justice (Farrington 1983, 2003; Farrington and Welsh 
2005, 2006; Sherman 2009, 2010; Sherman et al. 1997, 2006a, b) were searched. 
Third, forward searches were performed for works that cited seminal policing exper- 
iments (e.g., Sherman and Berk 1984; Sherman and Weisburd 1995; Clayton et al. 
1991; Ringwalt et al. 1991; Strang et al. 1999; Sherman et al. 2000; Weisburd and 
Green 1995). Fourth, bibliographies of past completed Campbell systematic reviews 
of police interventions were searched (Mazerolle et al. 2007; Weisburd et al. 2008a, b; 
Bowers et al. 2011; Braga et al. 2012). Fifth, hand searches of leading journals in the 
field were performed.* 

The searches were all completed between January 2012 and May 2012. Thus, 
the review only covers studies completed in 2011 and earlier. Sixth, after 
finishing the above searches and reviewing the identified studies, the list of 
studies meeting our eligibility criteria was emailed to leading criminology and 
criminal justice scholars knowledgeable in the area of policing experiments. 
These 129 scholars were defined as those who authored at least one study which 
appeared on our inclusion list, anyone involved with the National Academy of 
Sciences review of police research, and other leading scholars (list available 
upon request from authors). This helped to identify any missing studies. Finally, 
an information specialist was engaged at the outset of our review and at points 


? The following three search terms were used: randomized controlled trial AND police, randomized 
experiment AND police, and experiment AND police. 

? The following 15 databases were searched: Criminal Justice Periodical Index, Sociological Abstracts, 
Social Science Abstracts (SocialSciAbs), Social Science Citation Index, Arts and Humanities Search 
(AHSearch), Criminal Justice Abstracts, National Criminal Justice Reference Service (NCJRS) Abstracts, 
Educational Resources Information Clearinghouse (ERIC), Legal Resource Index, Dissertation Abstracts, 
Government Publications Office, Monthly Catalog (GPO Monthly), Google Scholar, Online Computer 
Library Center (OCLC) SearchFirst, CINCH data search, and C2 SPECTR (The Campbell Collaboration 
Social, Psychological, Educational and Criminological Trials Register). 

* These journals were: Criminology, Criminology & Public Policy, Justice Quarterly, Journal of Research in 
Crime and Delinquency, Journal of Criminal Justice, Police Quarterly, Policing, Police Practice and Research, 
British Journal of Criminology, Journal of Quantitative Criminology, Crime & Delinquency, Journal of 
Criminal Law and Criminology, and Policing and Society. Hand searches covered 1970-2011. 
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Fig. 1 Randomized controlled trials in policing, 1970-2011 (N=63) 


along the way in order to ensure that appropriate search strategies were used to 
identify the studies meeting the criteria of this review.” 


Search results 


Search strategies in the systematic review process generate a large number of citations 
and abstracts for potentially relevant studies that must be closely screened to deter- 
mine whether the studies meet the eligibility criteria (Farrington and Petrosino 2001). 
The screening process yields a much smaller pool of eligible studies for inclusion in 
the review. The search strategies produced 16,961 abstracts that included the search 
terms. The contents of these abstracts were reviewed for any suggestion of a 
randomized controlled trial of a police program. A total of 355 distinct abstracts 
were selected for closer review and the full-text reports, journal articles, and books for 
these abstracts were acquired and carefully assessed to determine whether the study 
met the eligibility criteria; 54 documents describing 63 eligible studies were identi- 
fied and included in this review (see separate list in reference section). 

Figure | presents the yearly counts of the dates that the 63 policing experiments 
were completed, as determined by key journal publication, final report, or dissertation 
completion dates. While field work was completed between 1963 and 1965, the first 
randomized controlled trial in policing was published in the British Journal of 
Criminology in 1970. This evaluation was conducted by University of Manchester 


> Ms. Phyllis Schultze of the Gottfredson Library at the Rutgers University School of Criminal Justice 
executed the initial abstract search and was consulted throughout on our search strategies. 
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researchers and tested a juvenile liaison scheme implemented by the Lancashire 
County Police Force (Rose and Hamilton 1970). The next policing experiment was 
not completed until 1979. Byles and Maurice (1979) used a randomized experimental 
design to test the effects of police and mental health collaborations to address family 
and personal problems of treated juveniles relative to traditional police arrest re- 
sponses in Hamilton, Ontario, Canada. 

While there were only two policing experiments completed during the 1970s, the 
pace of experimentation increased during the mid to late 1980s with an additional 
seven studies completed during the decade. The Minneapolis Domestic Violence 
Experiment, published in the American Sociological Review, was the first policing 
randomized controlled trial completed in the United States (Sherman and Berk 1984). 
As Fig. 1 reveals, the number of policing randomized controlled trials increased 
dramatically post 1990 with a total of 24 randomized experiments completed between 
1990 and 1999. Since 2000, a total of 30 policing randomized controlled trials have 
been completed. 

Table | presents the basic characteristics of policing randomized controlled trials 
completed between 1970 and 2011. Nearly 75 % (47 of 63) were completed in the 
United States. Eleven policing randomized controlled trials were completed in the 
United Kingdom, four were completed in Australia, and one was completed in 
Canada. The 63 eligible policing randomized experiments were implemented in 38 
different jurisdictions: the cities of London (UK), New York (NY), and Canberra 
(AUS) were the locations of four experiments each while Jersey City (NJ) and 
Minneapolis (MN) were the locations of three experiments each. Sherman (2010) 
has previously noted concentrations of crime and justice field experiments in partic- 
ular jurisdictions. Drawing on the history of randomized field experiments in med- 
icine and agriculture, Sherman (2010: 409) recognized the need to develop “field 
stations” where ongoing collaborations between academics and criminal justice 
practitioners could facilitate the growth of randomized experiments in criminology. 

Since many policing experiments were conducted in the US, it is not surprising 
that US government grant-making agencies and private foundations were the funding 
sources for the bulk of the eligible studies (Table 1). It is noteworthy that the US 
National Institute of Justice (NIJ) supported 37 policing randomized controlled trials, 
or nearly 60 % of the total, between the early 1980s and 2011. In their historical 
review, Farrington and Welsh (2006) observe that the use of randomized experiments in 
crime and justice studies goes through periods of “feast and famine.” NIJ deserves much 
credit for helping to initiate and sustain the increased use of randomized experiments in 
policing. As Farrington (2003, p. 220) observed, “the tenure of James K. ‘Chips’ Stewart 
as Director of the National Institute of Justice between 1981 and 1988 ushered in a new 
golden age of randomized experiments in American criminology.” 

More recently, the UK government has supported increased experimentation in 
England. The UK Home Office has supported a total of nine policing randomized 
controlled trials, with seven restorative justice randomized experiments completed 
during the 2000s (Angel 2005; Sherman et al. 2005, 2006a, b). Private foundations 
are also important sources of support for policing randomized controlled trials. The 
Jerry Lee Foundation provided support for 11 of the 14 restorative justice experi- 
ments identified in this review. The Smith Richardson Foundation has supported five 
randomized experiments in policing. 
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Table 1 Basic characteristics of 


policing randomized controlled Country 

trials 1970-2011, n=63 United States 
United Kingdom 
Australia 
Canada 


Intervention type 
Hot spots/crime places 
Restorative justice 
Domestic/family violence 
Drug abuse resistance education 
Juvenile interventions 
Stranger repeat offender apprehension 
Fear reduction 
Crime victim outreach 
Citizen feedback on services/information 
DNA crime solving 
TASER use 
Shift length hours 
Eyewitness identification 
Funding source* 
US National Institute of Justice 
Jerry Lee Foundation 
UK Home Office 
Smith Richardson Foundation 
AUS Criminology Research Council 
US National Institute on Drug Abuse 
US National Institute of Mental Health 


US Bureau of Justice Assistance 


Robert Wood Johnson Foundation 


US Community Oriented Policing Services 
Ford Foundation 

Canadian Health & Welfare Department 
North Carolina Department of Education 
Open Society Foundations 

JEHT Foundation 

Lily Endowment 


F Donner Foundation 
“These categories are not 


mutually exclusive as some Esmée Fairbairn Foundation 
experiments were supported Laura and John Arnold Foundation 
through grants from multiple No funding source identified 


funding sources 


US Office of Juv. Justice & Delinquency Prev. 


47 
ll 


Ree Se EF NY NY NYY FD 


wis 
— 


NNNN PB BRN OY 


Percent 
74.6 
17.5 
6.3 
1.6 
Percent 
22:2 
22.2 
20.6 
9.5 
6.3 
3.2 
3.2 
3.2 
3.2 
1.6 
1.6 
1.6 
1.6 
Percent 
58.7 
17.4 
14.3 
7.9 
6.3 
6.3 
3.2 
3.2 
3.2 
3:2 
1.6 
1.6 
1.6 
1.6 
1.6 
1.6 
1.6 
1.6 
1.6 
1.6 
6.3 


The results of our systematic review show the concentration of randomized controlled 
experiments in a small number of police policy areas. More than three-fourths of policing 
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randomized controlled trials have involved tests of hot spots/place-based policing strat- 
egies (14, 22.2 %), restorative justice schemes (14, 22.2 %), domestic/family violence 
interventions (13, 20.6 %), and drug abuse resistance education (six, 9.5 %). As detailed in 
Table 1, randomized controlled trials have also been used to evaluate a variety of other 
policing interventions such as juvenile delinquency reduction, stranger repeat offender 
apprehension, fear reduction, crime victim outreach, and citizen feedback on police 
service programs. 

Replications of highly influential experiments clearly contributed to the growth of 
policing randomized controlled trials during the study time period. For instance, the 
NIJ-supported Minneapolis Domestic Violence Experiment revealed that mandatory 
arrest for misdemeanor offenses worked best: it significantly reduced repeat offenses 
relative to mediation and separation approaches (Sherman and Berk 1984). The 
results of the experiment were very influential as many police departments adopted 
mandatory misdemeanor arrest policies and a number of states adopted mandatory 
misdemeanor arrest and prosecution laws (Sherman 1992). With funding from NIJ, 
randomized controlled trial replications in five US cities—Charlotte (Hirschel et al. 
1992), Colorado Springs (Berk et al. 1992), Omaha (Dunford et al. 1990), Milwaukee 
(Sherman et al. 1991), and Miami-Dade County (Pate and Hamilton 1992)—soon 
followed.° Subsequently, randomized controlled trials seemed to become a common 
design in the evaluation of police-led programs to address domestic/family violence 
problems. Beyond Minneapolis and its replications, our systematic review identified 
another seven randomized field experiments in this police policy area. 

Similarly, two influential randomized controlled experiments in Minneapolis ini- 
tiated during the late 1980s—Repeat Call Address Policing (Sherman et al. 1989a, b) 
and Hot Spots Patrol (Sherman and Weisburd 1995)—led to the development of 
additional hot spots/place-based policing randomized experiments (Braga and 
Weisburd 2010). These 12 subsequent hot spots/place-based policing experiments 
examined the impacts of varying intervention strategies (e.g., increased traditional 
enforcement, problem-oriented policing, place managers, and disorder policing), the 
efficacy of these approaches in different types of crime hot spots (e.g., general crime 
hot spots, violent crime hot spots, and drug markets), and whether these interventions 
generate spatial crime displacement or diffusion of crime control benefits effects (see 
Braga et al. 2012). 

A long-term research program has been explicitly managed to examine the impacts 
of restorative justice programs on a range of violent, property, and drunk-driving 
offenders in varying settings through a series of replications. Led by researchers at the 
University of Pennsylvania, University of Cambridge, and Australian National Uni- 
versity, the Jerry Lee Program on Randomized Controlled Experiments in Restorative 
Justice has completed 11 randomized controlled trials of police-involved restorative 
justice programs in Australia and the UK between 1995 and 2006 (for a review, see 


° The replications of the Minneapolis Domestic Violence Experiment in five other cities did not produce the 
same findings. In his review of those differing findings, Sherman (1992, p.19) identified four policy 
dilemmas for policing domestic violence: (1) arrest reduces domestic violence in some cities but increases 
it in others, (2) arrest reduces domestic violence among employed people but increases it among unem- 
ployed people, (3) arrest reduces domestic violence in the short run but can increase it in the long run, and 
(4) police can predict which couples are most likely to suffer future violence, but our society values privacy 
too highly to encourage preventive action. 
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Table 2. Lead authors with more 


ae . Lead Total Collaborators Mean per 
than one policing randomized 


controlled trial ane Re Te) al 
Lawrence 10 17 22 3.1 
Sherman 
Heather Strang 4 9 6 3.7 
Robert C. Davis 4 5 B} 1.4 
David Weisburd 3 7 15 2.6 
Anthony Pate 3 3 5 3.0 
Lorraine Green 2 4 8 25 
Mazerolle 
Caroline Angel 2 4 6 1:5 
Bruce Taylor 2 3 4 1.7 
Anthony Braga 2 2 6 3.0 
Franklyn Dunford 2 2 2 1.0 
Paul McCold 2 2 1 1.0 


Sherman and Strang 2007).’ In addition to the growing number of Jerry Lee-sponsored 
randomized experiments in this area, two police-led restorative justice randomized 
experiments have been completed in Bethlehem (PA) (McCold and Wachtel 1998) 
and one has been completed in Indianapolis (IN) (McGarrell et al. 2000). 

The published and unpublished reports for the 63 randomized policing experiments 
were authored by a total of 126 distinct individuals. However, a cursory review of the 
author list reveals that a few scholars were responsible for a bulk of the policing 
experiments. Further, these “multi-experiment” scholars tended to collaborate with a 
number of others on their experiments. Table 2 presents a list of 11 scholars who were 
the first authors of more than one randomized policing experiment. These 11 scholars 
led 36 randomized policing experiments (57.1 % of 63) and were authors (lead or 
otherwise) of 39 randomized policing experiments (61.9 % of 63). Lawrence Sherman 
was the most prolific experimenter in this group. He was the lead author of 10 policing 
experiments (15.8 % of 63) and was an author (lead or otherwise) of 17 policing 
experiments (26.9 % of 63). Sherman collaborated with a total of 22 other individuals 
on these 17 randomized experiments and averaged 3.1 collaborators per publication. 

Many of these 11 scholars collaborated with others identified in Table 2. For instance, 
Sherman most frequently collaborated with Heather Strang, as they jointly worked on 9 
restorative justice experiments together. Sherman also worked with David Weisburd 
(one experiment), Anthony Pate (two experiments), and Caroline Angel (two 
experiments)—who was his graduate student at the University of Pennsylvania. Robert 
C. Davis collaborated with David Weisburd on one experiment and Bruce Taylor on two 
experiments. David Weisburd collaborated with Sherman, Davis, Lorraine Green 
Mazerolle (one experiment), and Anthony Braga (one experiment). Among the ten 


7 The UK restorative justice randomized controlled trials were designed and implemented by Lawrence 
Sherman and Heather Strang. However, it is important to note here that Shapland et al. (2008) were 
assigned by the UK Home Office as independent evaluators to measure the results of the experiments. 
Given their seminal role in the development and execution of these studies, we credited Sherman and 
Strang reports and publications as the primary references for these randomized experiments. 
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other scholars in Table 2, Green Mazerolle, Braga, and Bruce Taylor were graduate 
students of Weisburd when he was a professor at the Rutgers School of Criminal Justice. 

At face value, this simple look at the most frequent lead authors of randomized 
policing experiments suggests that the professional network of scholars involved in 
these studies can be characterized by a small number of central players who are densely 
connected to others in the network. It also shows that some repeat experimenters were 
involved in randomized policing experiments as graduate students or were taught 
experimental methods in graduate school by leading police experimenters. We formally 
investigated the centrality of individual scholars and the importance of mentoring 
connections within the full network by using social network analysis. 


Social network analysis 


The importance of social relations in understanding human behavior is the focus of 
social network analysis (Scott 2000) and has witnessed an increase in criminology 
over past decade (McGloin and Kirk 2010; Papachristos 2011). We used social 
network analysis techniques to describe the professional network of n=126 scholars 
who completed randomized experiments in policing and to understand how 
mentoring and collaborative relationships connect these scholars across policing 
experiments. We also investigated how certain cliques of scholars and their protégés 
have contributed to the growth of policing experiments between 1970 and 2011. We 
used R and igraph software to create sociograms of co-author and mentoring social 
networks and to estimate network measures to describe the properties of these 
networks (R Core Team 2012; Csardi and Nepusz 2006). 

Following graph theory, social network data have two relevant units of analysis: 
the scholars (or “‘nodes) and the relationships among them (or “ties”). A graph or 
social network is thus defined formally as a set of nodes and the ties among them 
(Wasserman and Faust 1994). Ties between scholars were defined in two ways. First, 
scholars were connected when they had co-authored the main report of a policing 
randomized controlled trial. For instance, the Minneapolis Domestic Violence Ex- 
periment connected Sherman and Berk (1984) as co-authors. Second, we also 
connected scholars based on evidence of a previous mentoring relationship. The 
Colorado Springs replication experiment connected Berk et al. (1992) as co-authors 
and also connected Berk as a mentor to Campbell, Western, and Klap, who were his 
graduate students at the time the experiment was completed. Mentor—protégé 
relationships were determined by reading biographical sketches of authors on polic- 
ing experiment reports, reviewing curriculum vitae of study authors to determine 
whether they had been trained by senior experimenters as graduate students, and 
through direct communications (via email, phone, and face-to-face conversation) with 
study authors. These data were used to create three socio-matrices of relationships: a 
co-author matrix, a mentoring matrix, and a combined co-author/mentoring matrix. 
Table 3 provides the individual scholar associated with each numbered node in all 
sociograms discussed here. 

Figure 2 presents the sociogram of the co-authorship relationship matrix. This 
sociogram is characterized by one large component of 54 scholars (42.8 % of the 126 
scholars), 13 smaller components ranging in size from 3—9 scholars, four dyads, and 
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Table 3 Scholars in policing randomized experiments networks 


eaerANI Dn FW NHN 


41 


42. 


. GORDON ROSE 

. J. A. BYLES 

. LAWRENCE SHERMAN 
. ANTHONY PATE 

. PAUL LAVRAKAS 

. MARY ANN WYCOFF 

. WESLEY SKOGAN 

. MALCOLM KLEIN 

. SUSAN MARTIN 


. MICHAEL BUERGER 

. FRANKLYN DUNFORD 
. DAVID HUIZINGA 

. RICHARD CLAYTON 

. ANNE CATTARELLO 


. CHRISTOPHER RINGWALT 


. SUSAN ENNETT 

. ALLAN ABRAHAMSE 
. PATRICIA EBENER 

. PETER GREENWOOD 
. NORA FITZGERALD 


. DEAN COLLINS 

. DENNIS ROGAN 

. ELLEN COHN 

. JANELL SCHMIDT 

. PATRICK GARTIN 

. DAVID HIRSCHEL 

. IRA HUTCHINSON 

. ARTHUR LURIGIO 

. ALEC CAMPBELL 

. BRUCE WESTERN 

. RICHARD BERK 

. DENNIS ROSENBAUM 
. ROBERT FLEWELLING 


. SUSAN BAILEY 

. DAVID WEISBURD 

. ROBERT DAVIS 

. ANNETTE JOLIN 

. ROBERT FOUNTAIN 

. WILLIAM FEYERHERM 
. PAUL MCCOLD 

. COLLEEN KADLECK 
LORRAINE GREEN- 


MAZEROLLE 


43. 
44. 
45. 
. WILLIAM SPELMAN 

. HEATHER STRANG 

. GEOFFREY BARNES 

. JOHN BRAITHWAITE 

. JAMES PRICE 

. EDMUND MCGARRELL 
. KATHLEEN OLIVARES 

. KAY CRAWFORD 

. BRUCE TAYLOR 

. MATTHEW GIBLIN 

. CHERYL PERRY 

. KAREN MUNSON 

. KELLI KOMRO 

. KIAN FARBAKHSH 

60. 
61. 
62. 


JOHN ECK 
ANTHONY BRAGA 
ELIN WARING 


LINDA BOSMA 
MELLISA STIGLER 
SARA VEBLEN- 


MORTENSON 


63. 
64. 
65. 
66. 
67. 
68. 
69. 
70. 
71, 
72; 
723. 
74. 
13: 


76. 
77. 
7B. 
79. 
80. 
81. 
82. 
83. 
84. 


MELISSA STIGLER 
JULIE HORNEY 
WILLIAM WELLS 
CAROLINE ANGEL 
DANIEL WOODS 
SARAH BENNETT 
NANCY MORRIS 
ZILI SLOBODA 
RICHARD STEPHENS 
PEGGY STEPHENS 
SCOTT GREY 
BRENT TEASDALE 
RICHARD HAWTHORNE 


JOSEPH WILLIAMS 
AARON CHAFLIN 
JOHN ROMAN 
SHANNON REID 
JUSTIN READY 
WILLIAM SOUSA 
CHRISTOPHER KOPER 
ELIZABETH GROFF 
JERRY RATCLIFFE 


85. 
86. 
87. 
88. 
89. 
90. 
91. 
92. 
93. 
94. 
95. 
96. 
97. 
98. 
99. 


10 


= 


111 


115 
116 
117 


125 


126. 


100. 


102. 
103. 
104. 


105. 
106. 
107. 
108. 
109. 
110. 


TRAVIS TANIGUCHI 
CHRISTINE FAMEGA 
JOSHUA HINKLE 
BREANNE CAVE 
CYNTHIA LUM 
JULIE HIBDON 
EDWIN HAMILTON 
GREG JONES 
KAREN AMENDOLA 
GARY WELLS 
NANCY STEBLAY 
R.A. HAMILTON 

A. MAURICE 
DELBERT ELLIOTT 
KATHERINE WALDEN 
KATHLEEN HOLT 

. THOMAS KOSIN 
ANTHONY BACICH 
CHARLES DEAN 
RUTH KLAP 


DEANNA WILKINSON 
SHARON FRIEDMAN 
BENJAMIN WATCHEL 
JAN ROEHL 

JULIE WARTELL 
FRANK GAJEWSKI 


. NATALIE KROOVAND 
112. 
113. 
114. 


JUANJO MEDINA-ARIZA 
CHRISTOPHER MAXWELL 
LESLIE LYTLE 

. EDWARD MAGUIRE 

. NOVA INKPEN 

. DOROTHY NEWBURY- 


BIRCH 
118. 
119. 
120. 
121. 
122. 
123. 
124. 


BRENDA BOND 
JESSE MARQUETTE 
CARLY KNIGHT 
MICHAEL AULT 
JENNIFER WOOD 
LINDA MEROLA 
MEGHAN SLIPKA 

. JENNIFER DYSART 
PAUL QUINTON 
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Fig. 2. The social network of co-authors in policing randomized experiments 


three isolates (Klein—node eight, Giblin—node 55, Quinton—node 126). The 
scholars in the large clique are responsible for the completion of 38 randomized 
controlled trials in policing (60.3 % of the 63 RCTs). The geodesic is the shortest 
distance between two nodes (Wasserman and Faust 1994). The largest component 
has an average geodesic of 3.3. As such, nearly two-thirds of the policing 
randomized experiments were completed in a single connected network where 
people are, on average, only three handshakes away from each other. Certain 
scholars in this large component are connected to high numbers of other scholars. 
As mentioned earlier, Sherman (node 3) has co-authored with 22 other scholars, 
Weisburd (node 35) co-authored with 15 other scholars, and Green Mazerolle 
(node 42) co-authored with eight other scholars. 

Figure 3 presents the sociogram of the mentoring network with the isolates excluded. 
Out of the 126 scholars comprising the co-author network, 46 scholars (36.5 %) were 
connected either directly or indirectly through a mentoring relationship. There were 36 
randomized policing experiments (57.1 % of 63 RCTs) that included a graduate student 
on the final report or were completed by a scholar who was previously mentored by a 
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Fig. 3. The social network of mentors and protégés in policing randomized experiments 


senior experimentalist. The arrows represent a directional relationship that ema- 
nates from the mentor to the protégé. The mentoring sociogram is characterized 
by one large component of 37 scholars, one triad, and three dyads. The large 
clique is anchored by two highly central nodes—Weisburd and Sherman. Inter- 
estingly, both of these highly influential experimental criminologists were 
mentored by Albert J. Reiss, Jr. when they were doctoral students in sociology at Yale 
University. In this regard, Reiss could be considered the “godfather” of randomized 
experiments in policing. 

Weisburd (node 35) has mentored 14 other scholars in this component. Weisburd 
coauthored policing experiments with eight of his graduate students and trained another 
six graduate students who went on to execute policing experiments on their own. It is 
also important to note that four of Weisburd’s protégés have, in turn, mentored graduate 
students who were involved in the completion of a policing randomized controlled trial. 
These scholars include Green Mazerolle (node 42), Braga (node 44), Groff (node 83), 
and Lum (node 89). For instance, Lum completed a randomized policing experiment 
with Hibdon (node 90) and Cave (node 88) who also received training from Weisburd in 
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experimental methods at George Mason University. Weisburd, Lum, Hibdon, and Cave 
worked as staff at the Center for Evidence Based Crime Policy, a research center that 
promotes the use of randomized experiments in criminal justice policy evaluation. 
Wilkinson (node 105) received training from Weisburd while a doctoral student at 
Rutgers but completed a randomized experiment with Rosenbaum (node 32) while 
she was a masters student at the University of Illinois, Chicago. 

Sherman (node 3) has mentored 10 other scholars in this clique. Five of his 
protégés were jointly mentored by Strang (node 47) as part of their randomized 
experiments in restorative justice policing in Australia and the UK. Geoffrey Barnes 
(node 48) received training from and worked on randomized experiments as a 
graduate student with Sherman, Strang, and Berk (node 31). As of 2011, only one 
of Sherman’s protégés had subsequently mentored graduate students who have been 
involved in the completion of a randomized policing experiment. John Eck (node 43) 
was a senior scholar with many years of research experience when he completed his 
PhD under Sherman’s supervision at the University of Maryland in 1994. While 
pursuing her PhD at the University of Cincinnati, Famega (node 86) studied under 
Eck and Green Mazerolle (node 42) and published articles with both. Famega later 
conducted a randomized controlled trial evaluating the efficacy of broken windows 
policing at crime hot spots with Weisburd (node 35). 

We also examined the number of randomized controlled trials completed during 
the study time period by mentored scholars after the completion of their PhD degrees 
and without the direct involvement of their mentors. This search included randomized 
controlled trials evaluating interventions in other criminal justice policy areas.® We 
found that 12 mentored scholars went on to be involved in the production of 23 
additional randomized controlled trials. Thirteen of these randomized experiments 
were in the policing field and are included in this study; six involved randomized 
controlled trials in other criminal justice policy areas. Some of these mentored 
scholars were involved in multiple randomized controlled trials. For instance, Bruce 
Taylor (node 54) was involved in ten randomized experiments and Christopher 
Maxwell (node 113) was involved in three randomized experiments. Anthony Braga 
(node 44), Lorraine Green Mazerolle (node 42), Elizabeth Groff (node 83), and 
Christopher Koper (node 82) were involved in two randomized experiments, each 
post-completion of their PhD degrees and independent of their mentors. 

Figure 4 presents the sociogram of the combined co-author and mentoring net- 
works. This sociogram is characterized by one large component of 68 scholars 
(54.0 % of the 126 scholars), 11 smaller components ranging in size from 3-9 
scholars, three dyads, and three isolates. The largest component accounted for 43 
randomized policing experiments between 1970 and 2011 (68.3 % of 63 RCTs). 
More importantly, this component dominated the growth in policing randomized 
experiments during the study time period with zero RCTs in the 1970s (0 % of two 
completed that decade), six in the 1980s (85.7 % of seven completed that decade), 17 
in the 1990s (70.8 % of 24 completed that decade), and 20 in the 2000s (66.7 % of the 


* This supplemental search was completed by taking each mentored scholar’s name and replicating the 
abstract search procedures described earlier in the systematic review methods section. For example, we 
combined “Justin Ready” with “randomized controlled trial”, “randomized experiment”, and “experiment” 
and searched for these keyword terms in the 15 online abstract databases. Abstracts were screened and 
potentially eligible studies were obtained and reviewed. 
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Fig. 4 Combined social network in policing randomized experiments 


30 completed that decade plus 2011). The number of policing experiments involving 
protégés mentored by senior experimenters also grew from one randomized experi- 
ment in the 1980s to 13 randomized experiments in the 1990s to 20 randomized 
experiments in the 2000s (plus 2011). 

Consistent with the separate co-author and mentoring networks, the combined 
network can be characterized as anchored by a small number of highly central 
scholars. Visually, this can be seen in Fig. 4, where these highly central actors are 
at the center of small clusters within the network—for example, Weisburd (node 25) 
connects small clusters of scholars through ties with Buerger (10), Groff (83), 
Wilkinson (node 105), and others. Figure 5 presents the distribution of the number 
of network ties or “degrees” for the 126 scholars that comprise the entire policing 
experiment network; degrees are considered measures of activity (in this case, co- 
authoring and mentoring). The mean number of degrees per scholar in the network 
was 5.4 with a standard deviation of 5.1 degree. However, the distribution is skewed 
to the right with a quarter of the scholars having a small number of degrees (31 
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scholars, 24.8 % of 125, had two or fewer ties) and two scholars with 32 and 39 
degree (Weisburd and Sherman, respectively). The right-skewed distribution of 
network ties is typical of other types of social networks (Scott 2000). For instance, 
Papachristos et al. (2012) examined the ties among Boston gang members and found 
that many gang members had a small number of ties to other individuals and a few 
had a very large number of ties across the gang network. 

The number of ties is a very basic measure of centrality within a network. A finer 
grained measure of centrality within a network examines the “betweenness” of 
particular nodes (Freeman 1979: 237). As mentioned earlier, the geodesic distance 
refers to the shortest path between two nodes (Knoke and Kuklinski 1982), where the 
distance between two nodes n; and n, is measured simply as d(i, j). “Betweenness 
centrality” is a measure of a node’s centrality in a network equal to the number of 
shortest paths from all vertices to all others that pass through that node (Freeman 
1977). In other words, a node with a high betweenness centrality lies on a greater 
number of shortest paths—he or she is literally “between” a greater number of nodes. 
Betweenness centrality measures the “brokerage” of particular nodes within a net- 
work and quantifies the flow of communication through particular actors. In the 
policing experiment network, it provides a measure of the extent that any individual 
scholar is in the middle of—and perhaps brokers—influence, ideas, and resources 
within then network. 
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Table 4 Betweenness centrality: 


: as Rank/scholar Normalized betweenness 
top five scholars in policing random- 
ized experiments network ; ; 
1. David Weisburd 0.2002 
2. Lawrence Sherman 0.1554 
3. Lorraine Green Mazerolle 0.0252 
4. Richard Berk 0.0251 
5. Elizabeth Groff 0.0250 


Table 4 presents the top five scholars in the policing randomized experiment 
network according to the normalized betweenness centrality measure. We normalized 
the measure for ease of interpretation; values range from 0 to 1 with higher numbers 
indicating greater centrality relative to other network members (Knoke and Kuklinski 
1982). The absolute value of the measure is less important than the relative value. Not 
surprisingly, Weisburd (0.2002) and Sherman (0.1554), respectively, are the two most 
central members of the network when paths between other nodes are considered. 
Indeed, no other scholar in the network comes close to Weisburd and Sherman as a 
hub of connections to other scholars. The difference between the top two scholars and 
the third and fourth ranked scholars (Green Mazerolle, 0.0252; Berk, 0.0251) is very 
large. The magnitude of Weisburd’s normalized betweenness centrality is 8 times 
larger than the normalized betweenness centrality of Green Mazerolle and Berk; by 
this same measure, Sherman’s score is 6.2 times larger than the scores of Green 
Mazerolle and Berk. This comparison is not intended to take away from the important 
contributions of Green Mazerolle, Berk, or any of the other scholars in the network. 
Rather, it demonstrates the vital importance of two very central scholars in advancing 
experimentation in the policing field. 


Discussion and conclusions 


The number of randomized experiments in policing increased dramatically between 
1970 and 2011. This growth was especially pronounced in the 1990s and 2000s, 
which included the completion of 54 policing experiments (85.7 % of the population 
of 63 RCTs). The major growth areas of experimental inquiry have included crime 
hot spots, restorative justice, domestic/family violence, and drug abuse resistance 
education. The growth of police experiments in these areas and other policing areas 
can be characterized as relying on support from public funding agencies, such as the 
U.S. National Institute of Justice, and private funding agencies such as the Jerry Lee 
Foundation. These experiments also required willing host criminal justice organiza- 
tions with some agencies serving as the sites for multiple randomized experiments. 
One of the most striking aspects of the growth in policing randomized experiments is 
the small number of scholars who carried out the work; that is, rather than grown by 
spreading to a large number of adaptors, randomized experiments grew by the produc- 
tivity of a small and highly active connected network of collaborators and students. The 
professional network is dominated by a vital few experimentalists who account for the 
bulk of policing experiments and have been very active in mentoring the next generation 
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of experimental policing scholars. For instance, during the study time period, Lawrence 
Sherman and David Weisburd have been involved in the completion of a combined 23 
policing randomized experiments (36.5 % of 63 RCTs) and have mentored a cadre of 
young scholars who have gone on to complete additional randomized experiments. 
Indeed, without Sherman, Weisburd, and a handful of other committed scholars, there 
would be very few randomized experiments in policing (and the observed network 
would essentially fall apart). 

Our investigation of one area of experimental criminology (policing) suggests that 
the area of experimental criminology is beginning to gather the necessary human 
resources—through the mentoring of graduate students in experimental methods—to 
potentially shift scientific paradigms in mainstream criminology towards increased 
experimentation. The potential for change seems great as this human capital is layered 
on the social capital observed in the observed policing network. It is also important, 
however, to note that the pool of experimental criminologists is very small. While there 
are non-DEC-member scholars who use randomized experiments in their crime and 
justice research agenda, slightly more than 7 % of ASC members also participate in its 
DEC. Relative to the broader ASC membership, the professional network of scholars 
who have completed policing randomized experiments can be characterized as a “small- 
world” network (see Watts 1999; Watts and Strogatz 1998).°? In other words, this 
network contains densely connected pockets of activity with several individuals (e.g., 
Sherman and Weisburd) who expand outward from these clusterings. The challenge to 
experimental criminologists is to ensure that their small world enhances its connections 
to mainstream criminology and other related social science fields. 

We believe the best way to increase the number of experimental criminology converts 
is to increase the number of graduate students who are involved in randomized experi- 
ments and trained in the classroom by seasoned experimentalists. As suggested by Kuhn 
(1962) and Merton (1968), criminology and criminal justice graduate students represent 
the next generation of experimental scholars who will maintain the heritage of methods, 
systems of thought, and processes of success currently being maintained by the vital few. 
Graduate students are critically important to the continued proliferation of randomized 
experiments in criminology and to increasing the share of ASC members who use 
experimental methods in their research agendas. The DEC currently seeks to stimulate 
graduate student interest through e-mail mentoring programs, student paper awards, and 
young experimental scholar awards (for recent PhD graduates who exhibit great promise). 

The vital few experimental criminologists can also increase their numbers by en- 
couraging the “useful many” (Juran 1951) criminologists who do not currently use the 
method in their research to add randomized experiments as an important part of their 
methodological tool kit. There are many ways experimental criminologists have 
attempted to achieve this goal. To increase interest in randomized experiments, exper- 
imental criminologists have written persuasively in popular journals and edited volumes 
that randomized experiments enhance scientific quality, evidence-based policy, causal 
inference, and liberty (Farrington 2003; Sherman 2009, 2010; Weisburd 2003). DEC 


° In fact, the observed degree distribution in Figure 5 is one of the key properties of formal small-world 
graphs; the other two properties being a graph’s clustering coefficient (C) and average path length (L). Also, 
consistent with the properties of small-world graphs, initial tests suggest that the total network in Fig. 4 has 
a high clustering coefficient (C=0.606) and a short average path length (L=2.91). 
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also holds workshops on experimental methods and hosts a series of professional 
networking events at the annual ASC conference; non-DEC-members can sign up for 
the workshops and attend these events. 

Also important, experimental criminologists can collaborate with other scholars in 
the development and execution of randomized controlled trials. For instance, begin- 
ning in 2008, Philip J. Cook, a well-respected economist with no previous back- 
ground in randomized experiments, partnered with Jens Ludwig and Anthony Braga, 
two experienced experimenters, on his first randomized controlled trial evaluating a 
prisoner reentry program operating in Milwaukee, Wisconsin (Cook et al. 2012). 
Cook is now involved with several additional randomized experiments in Chicago 
and was recently elected a Fellow of the Academy of Experimental Criminology. 

Despite the relatively small number of scholars completing randomized controlled 
trials, experimental criminology seems to be on an upward trajectory within the broader 
field of criminology. Indeed, the proliferation of randomized experiments over the last 
three decades (Farrington and Welsh 2006; Welsh et al. 2013) confirms that an increas- 
ing number of criminologists recognize the advantages of the design in testing theories 
and criminal justice policy interventions. Our social network analysis of randomized 
experiments in policing reveals a dynamic network of highly productive academics who 
are training and mentoring new scholars in experimental methods. In turn, these new 
scholars are conducting randomized experiments independent of their mentors. These 
are necessary conditions to create a more central role for experimentation in criminol- 
ogy. The future looks bright for experimental criminology. 
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